-
Notifications
You must be signed in to change notification settings - Fork 141
RFC: ArrayBuffer support in TurboModules #947
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
RFC: ArrayBuffer support in TurboModules #947
Conversation
Firstly - congrats on a very thorough and well written RFC!
It's my understanding that the JS engine and JSI itself isn't thread-safe either.
I think that could be a nice addition indeed. I'd actually imagine a read-only variant would be used in most cases 🤔
Personally - I'd apply the 80-20 rule here and go for the least amount of work bringing most of the value and stick with the basic sync support. Did you consider "views"? ( |
i really like this. for real-time media or GPU pipelines, say camera ---> ML ---> WebRTC, efficient ArrayBuffer bridging would make a world of difference. it'd enable moving small binary payloads(LUTs, masks, uniform buffers) without having to serialize or clone data just to cross the bridge. a few things worth clarifying though:
|
Thanks ❤️
Agree with that.
I agree with that, but after deeper investigation I couldn't find an easy and clean way to achieve that. One way would be to create a read-only view over the buffer when processing it, but that's on the developers. Also there is an active TC39 proposal for an Immutable ArrayBuffer which would provide a standardized, runtime-enforced way to prevent modifications to the buffer contents.
👍
My idea is to have this solution type-agnostic as the underlying native classes, such as |
The semantics around memory ownership seem to deviate from the spec of ArrayBuffer in regards to transferring/detaching. I'm not sure of what the material consequences are, especially with existing code that handles ArrayBuffers, but it seems like this could break developer expectations in many ways. I'd expect the buffer to be moved, not borrowed, when passing between JS and native (in both directions).
This problem goes away if ownership is moved to the receiving thread. You shouldn't be able to even read an ArrayBuffer from multiple threads.
So in summary and to answer this unresolved question, I would say absolutely yes. And to do so by using moves and not borrows. |
Thanks @tom-sherman for your input! Regarding this:
I agree that moving (transferring ownership) an ArrayBuffer coul be fundamentally safer and cleaner than borrowing it. However, the primary technical challenge remains: the current JSI and Hermes Runtime implementations do not expose a dedicated API for "detaching" an ArrayBuffer from the JavaScript side. Without true detachment, the only immediate way to transfer ownership is by "moving" the underlying buffer to the native thread and extending its lifetime accordingly. This addresses the memory management aspect but has a critical flaw:
I currently do not see a clean, safe path to fully implement buffer transfers that invalidate the JS reference. Achieving this requires dedicated changes to both the JSI specification and the underlying Hermes engine to introduce a proper detachment mechanism. Since I don't have a deep expertise in this topic, output from more experienced developers is really welcome and highly appreciated. |
I don't have any expertise as to how to solve the invalidation of JS references in Hermes and JSI, but I wanted to add another voice highlighting the importance of ownership transfer. As far as I am aware, JS does not have other instances where a developer needs to think about thread-safety - it is always thread safe by default. When working with threads (e.g. a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this RFC. Excited to get this going. Leaving some comments for consideration when it comes to the RFC document itself.
```ts | ||
export interface Spec extends TurboModule { | ||
getBuffer(): ArrayBuffer; | ||
processBuffer(buffer: ArrayBuffer): void; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we extend the RFC with some notes on asynchronous functions? I know there was discussion beforehand whether to implement asynchronous code in the first PR, however, I would treat that separate from the RFC itself, which could cover broader use case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I unnecessarily coupled this RFC with the first PR in my head. Asynchronous functions are mentioned a bit later, but they should be outlined here as well as this is the final API we would like to have.
|
||
## Motivation | ||
|
||
TurboModules currently lack a first-class way to represent `ArrayBuffer` end-to-end in Codegen, which forces developers to rely on copies, ad-hoc platform bridges, global helpers, or external libraries. This hurts performance for binary-heavy use cases such as media data or ML tensors, and it increases implementation complexity. The expected outcome is a cross-platform contract that lets JS and native pass binary data with minimal copying. Codegen should be able to generate working code for the `ArrayBuffer` type on every platform. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would say current workaround is typically working directly with JSI, which has its own advantages and disadvantages. Great example here is prior art by Marc, who did a lot of this manually before migrating over to Nitro Modules. I also think a lot of Software Mansion libraries go with C++ and work directly with JSI for that reason as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup - both react-native-mmkv (see MMKVManagedBuffer.h
) and react-native-vision-camera (see SharedArray.h
) use raw JSI access to expose ArrayBuffer
data. In VisionCamera I kinda overcomplicated it for a while with TypedArray support (that's primarily for Uint8Array
), but still - it's a lot of effort to get right, and even tho I added a ton of code for that, it still has it's bugs. Like thread-safety.
Also; react-native-fast-tflite (see TypedArray.h
) and vision-camera-resize-plugin (see ResizePlugin.mm
) are using raw JSI ArrayBuffers.
In Nitro, i finally solved all of that - so MMKV benefits from that ✨ for free ✨, but VisionCamera is not on Nitro yet.
I can definitely understand the Motivation here, lol.
|
||
For example, several important use cases are currently difficult to implement efficiently while working with TurboModules: | ||
|
||
- **Real-time media streaming**: A native video decoder could stream frames directly to a JavaScript-based player component. Without zero-copy `ArrayBuffer`s, each frame would need to be copied, leading to significant performance overhead and potential frame drops. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think great example here would be to use Blob Manager as an example (and likely first candidate to migrate over, once this lands).
Broadly speaking working with binary data in general.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great catch, will add a Blob Manager example to the next version of the RFC.
- **Machine Learning**: On-device ML models often require passing large tensors between native inference engines and JS. Copying this data can be a major bottleneck, especially for real-time applications like video analysis. | ||
- **High-performance networking**: Applications that handle large binary payloads over WebSockets or other protocols (e.g., financial data streams, real-time gaming) may be forced into inefficient data conversion, which adds CPU and memory pressure. | ||
|
||
By providing a first-class `ArrayBuffer` type support to TurboModules, this RFC will unblock these and other performance-sensitive areas, making it possible to develop faster more efficient applications for React Native. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another important indication here is that first-class support for ArrayBuffers will enable support of web spec compatible primitives (such as Blob or File). It is also great way to enable interoperability across different module frameworks.
For example, Expo currently has its own Blob implementation (based on ArrayBuffers, which they support). If React Native has same level of support, libraries relying on each will be able to work interchangeably, which is great.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion - will expand this section accordingly.
That said, it's worth noting that (unless I'm mistaken) this is already how JSI, Expo Modules, and Nitro Modules handle ArrayBuffers today. On one hand, aligning with existing community behavior might make sense for practical and compatibility reasons. On the other hand, once this behavior becomes part of the core, its reach and visibility will likely expand far beyond those ecosystems, making the current de facto behavior less relevant over time. Leaving this as an open question and summoning a few folks from the community for feedback! |
To conclude, supporting only simple function input/output parameters is straightforward, but extending that support to cover all Codegen functionalities across every platform is significantly more complex. However, integrating Promises and asynchronous operations may be crucial for supporting binary-heavy use cases. This leads to the question: | ||
|
||
> [!IMPORTANT] | ||
> Should this RFC focus on introducing only the basic and most valuable synchronous support for ArrayBuffer, or should it aim for full coverage of all possible Codegen use cases, including asynchronous operations, despite the higher complexity and the impact on more files (especially on Android)? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my opinion, the RFC should aim to cover all use cases and describe the complete implementation plan. We can then approach it incrementally, breaking the work into smaller, manageable PRs. It might also make sense to update the Adoption Strategy section with a detailed roll-out plan that outlines the milestones we’ll follow once this RFC is approved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea - will do that!
Thanks for putting this together. Generally, I'm very aligned supporting a type-safe abstraction over the existing ArrayBuffer support in JSI, and there seems to be plenty of use-cases where this would become a good way forward to unlock cheaper data sharing between JS and native. I agree with the concerns around thread-safety expressed in this thread here. Is there any prior art we can reference? Is the operation model similar to SharedArrayBuffer? Should we consider Atomics as a complementary but necessary capability here? Making this fully support async JS to native invocation calls will likely increase complexity, as it would require us to keep the JS object alive for the duration of the native memory reference, but not impossible. Alternatively, we'd need to make it really obvious that ArrayBuffer args can only be used in sync calls through codegen. |
|
||
#### Java | ||
|
||
Java class `java.nio.ByteBuffer` can be constructed using JNI function `jobject NewDirectByteBuffer(JNIEnv* env, void* address, jlong capacity)`. This function returns a direct instance of direct `java.nio.ByteBuffer` referring to the block of memory starting at the memory address `address` and extending `capacity` bytes. What is important, direct buffers don't deallocate the memory on destruction, what is desired in our case since JS is responsible for that. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would recommend we use the existing fbjni abstraction here: https://github.com/facebookincubator/fbjni/blob/main/cxx/fbjni/ByteBuffer.cpp#L84
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missed that one - thanks!
When passing an `ArrayBuffer` to native code, it should always be treated as "borrowed" or "non-owning": JS owns the ArrayBuffer's memory and the JS GC is responsible for freeing it. Native code should access the passed memory only for the duration of the synchronous call. | ||
|
||
The same rules apply when passing buffers from native to JS: native code remains the owner of the allocated memory and may expose zero-copy buffers to JS. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this so? createArrayBuffer
uses a shared_ptr
so it should be feasible to implemented shared ownership semantics here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed - ownership can transfer from native to JS here, that's perfectly fine. We use this in a lot of Nitro libraries.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought that aligning the ownership strategy for both directions would make sense - but of course you are right, it would work fine for case of "native to JS" direction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like this RFC - adding ArrayBuffer
to react-native core seems like a no-brainer.
I can see that some of this was inspired by Nitro, where we already have nitro::ArrayBuffer
(see the ArrayBuffer
docs).
I remember implementing ArrayBuffer
s wasn't super easy, as a lot of work went into the nitro::ArrayBuffer
primitives - I think I learned a lot about how this works - I can help with the implementation if any questions arise.
Nitro has owning and non-owning ArrayBuffers, where owning is a native one (we can unwrap that later again), and non-owning is one that was created in JS, or somewhere else.
PROBLEM: If an owning ArrayBuffer (aka one that was created in native) gets passed to JS, and then back from JS to native, the information that it is a native ArrayBuffer gets lost because we don't have .isMutableBuffer()
and .getMutableBuffer()
in JSI.
I created a feature request for this here: facebook/hermes#1578 - @tmikov said he thinks it's a good idea - I even created a PoC PR that implements this (facebook/hermes#1733), but it's not fully finished - might need a little bit of help here from the Hermes team.
In Nitro, I have a workaround for this by simply attaching a NativeState
of the jsi::MutableBuffer
pointer to my object (https://github.com/mrousavy/nitro/blob/cffffcb91f90fe4823fa7410bdb39c51c4c99125/packages/react-native-nitro-modules/cpp/jsi/JSIConverter%2BArrayBuffer.hpp#L81-L83) - works very well 😄
But; this means it would not detect owning ArrayBuffers that were created natively in TurboModules, since TurboModules will likely not use nitro::MutableBufferNativeState
lol. Not a biggie, it'll just be non-owning.
We need the Hermes PR for native support for that.
|
||
### Memory ownership | ||
|
||
When passing an `ArrayBuffer` to native code, it should always be treated as "borrowed" or "non-owning": JS owns the ArrayBuffer's memory and the JS GC is responsible for freeing it. Native code should access the passed memory only for the duration of the synchronous call. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems like this terminology is from Nitro 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I sure did read your documentation and source code before starting working on this 😄
|
||
#### Java | ||
|
||
While in C++ and Objective-C data can be easily shared between JS and Native, Java stores the data as a `folly::dynamic` map on the Native side. The `folly` library has support for data buffers (class `IOBuf`). This means that JS buffers can be stored on the Native side, but its implementation will be more challenging. Moreover, classes responsible for storing variables, such as `NativeMap` or `NatviveArray`, have a rich inheritance tree and are widely used across the JNI files. Adding storage for buffers to them will require changes to a large number of `ReactAndroid` JNI and Java/Kotlin files. These changes are required to add support for e.g. Promises or Structs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While in C++ and Objective-C data can be easily shared between JS and Native, Java stores the data as a `folly::dynamic` map on the Native side. The `folly` library has support for data buffers (class `IOBuf`). This means that JS buffers can be stored on the Native side, but its implementation will be more challenging. Moreover, classes responsible for storing variables, such as `NativeMap` or `NatviveArray`, have a rich inheritance tree and are widely used across the JNI files. Adding storage for buffers to them will require changes to a large number of `ReactAndroid` JNI and Java/Kotlin files. These changes are required to add support for e.g. Promises or Structs. | |
While in C++ and Objective-C data can be easily shared between JS and Native, Java stores the data as a `folly::dynamic` map on the Native side. The `folly` library has support for data buffers (class `IOBuf`). This means that JS buffers can be stored on the Native side, but its implementation will be more challenging. Moreover, classes responsible for storing variables, such as `NativeMap` or `NativeArray`, have a rich inheritance tree and are widely used across the JNI files. Adding storage for buffers to them will require changes to a large number of `ReactAndroid` JNI and Java/Kotlin files. These changes are required to add support for e.g. Promises or Structs. |
|
||
#### Java | ||
|
||
While in C++ and Objective-C data can be easily shared between JS and Native, Java stores the data as a `folly::dynamic` map on the Native side. The `folly` library has support for data buffers (class `IOBuf`). This means that JS buffers can be stored on the Native side, but its implementation will be more challenging. Moreover, classes responsible for storing variables, such as `NativeMap` or `NatviveArray`, have a rich inheritance tree and are widely used across the JNI files. Adding storage for buffers to them will require changes to a large number of `ReactAndroid` JNI and Java/Kotlin files. These changes are required to add support for e.g. Promises or Structs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While in C++ and Objective-C data can be easily shared between JS and Native, Java stores the data as a
folly::dynamic
map on the Native side.
Yea I found the same issue a while ago - folly is quite cool but wrapping everything in dynamic probably has to go at some point in the future.
|
||
### Memory ownership | ||
|
||
When passing an `ArrayBuffer` to native code, it should always be treated as "borrowed" or "non-owning": JS owns the ArrayBuffer's memory and the JS GC is responsible for freeing it. Native code should access the passed memory only for the duration of the synchronous call. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When passing an `ArrayBuffer` to native code, it should always be treated as "borrowed" or "non-owning": JS owns the ArrayBuffer's memory and the JS GC is responsible for freeing it. Native code should access the passed memory only for the duration of the synchronous call. | |
When passing an `ArrayBuffer` that was created in JS to native code, it should always be treated as "borrowed" or "non-owning": JS owns the ArrayBuffer's memory and the JS GC is responsible for freeing it. Native code should access the passed memory only for the duration of the synchronous call. |
Let's clarify this a bit here; only if it was created in JS, we don't have a std::shared_ptr<jsi::MutableBuffer>
.
If it was created in native, we can technically unwrap the native buffer again and can assume ownership safely. But; see my other discussion for more info on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for clarification!
Proposal: Adding first-class
ArrayBuffer
support to Codegen and TurboModules to enable zero-copy binary data exchange between JavaScript and Native modules.View the rendered RFC